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ABSTRACT 



A study demonstrated the positive impact of school ownership 
of the Accelerated Reader (AR) technology-based literacy program on 
attendance and standardized test scores at a representative sample of 2,500 
elementary, middle, and high schools. These schools were compared with 
approximately 3,500 schools of similar geographic and demographic 
characteristics that did not own the software. A comparative analysis of data 
revealed: (1) statistically significant evidence that, on virtually every 

subject test (including reading, writing, math, science, and social studies), 
a majority of schools that owned AR performed better than socioeconomically 
comparable non-AR schools; (2) a statistically significant majority of 
AR-owning schools also had higher attendance rates than their non-AR peers; 

(3) gains in academic performance increase with the length of time schools 
own AR-- schools that have owned AR for 2 or more years are 59% more likely to 
show test performance above the median for their control group; (4) analysis 
of AR 1 s effectiveness in different metropolitan settings indicates that AR is 
by far most influential in urban schools and in low socioeconomic 
environments; and (6) increased performance of AR-owning schools does not 
vary with the relative availability of microcomputers at those schools, 
indicating that the effect is not related simply to increased use of 
technology. Findings suggest that AR has a positive effect on student 
academic performance, especially for socioeconomically disadvantaged children 
in urban areas. Together with the results of previous Institute studies, this 
is compelling evidence that AR is an effective tool in stimulating increased 
reading, and that increased reading will lead to higher attendance rates and 
greater academic success. (Includes 18 notes and 6 tables of data.) 
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Impact of the Accelerated Reader® 
Technology-Based Literacy Program on 
Overall Academic Achievement and 
School Attendance 

by Terrance Paul, Darrel VanderZee, Tom Rue, and 
Scott Swanson 

Abstract 

This study demonstrates the positive impact of school ownership of the Accelerated 
Reader (AR) technology-based literacy program on attendance and standardized test 
scores at a representative sample of 2,500 elementary , middle , and high schools. These 
schools were compared with approximately 3,500 schools of similar geographic and 
demographic characteristics that did not own the software. This analysis used data from 
Advantage Learning Systems , Inc. (publishers of AR), Quality Education Data , Inc., 
and state educational agency statistics. 

A comparative analysis of this data produced the following results: 

• Statistically significant evidence that, on virtually every subject test ( including 
reading, writing, math, science, and social studies), a majority of schools that 
owned AR performed better than socioeconomically comparable non-AR schools. 

• A statistically significant majority of AR-owning schools also had higher 
attendance rates than their non-AR peers. 

• Gains in academic performance increase with the length of time schools own AR. 
Schools that have owned AR for two or more years are 59 percent likely to show 
test performance above the median for their control group. 

• Analysis ofAR’s effectiveness in different metropolitan settings indicates that AR 
is by far most influential in urban schools and in low socioeconomic environments. 

• The increased performance of AR-owning schools does not vary with the relative 
availability of microcomputers at those schools, indicating that the effect is not 
related simply to increased use of technology. 

Based on these results, the report concludes that AR has a positive effect on student 
academic performance, especially for socioeconomically disadvantaged children in urban 
areas. Together with the results of previous Institute studies, this report provides 
compelling evidence that AR is an effective tool in stimulating increased reading, and 
that increased reading will lead to higher attendance rates and greater academic 
success. 
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Introduction 

This was a large-scale study of the effectiveness of Advantage Learning 
Systems’ Accelerated Reader technology-based literacy program. It was the 
fourth in a series of studies on reading completed by the Institute for Academic 
Excellence. Our objective was to determine whether the Accelerated Reader, 
which has been purchased by more than 32,000 schools nationwide, has a 
measurable, demonstrable effect on academic achievement and school 
attendance rates. The Accelerated Reader, or AR, is a program for literature- 
based reading practice in K-12 classrooms. As described below, AR helps 
teachers motivate students to read more books, while enabling teachers to 
evaluate, monitor, and record student progress. It provides continuous 
assessment and accountability for literature-based reading. 

The Institute for Academic Excellence was founded in 1993 by Judith and 
Terrance Paul, creators of the Accelerated Reader and owners of Advantage 
Learning Systems. The Institute develops instructional systems to improve 
academic achievement in K-12 schools, and trains educators in implementing 
these systems. The Institute’s Reading Renaissance program incorporates AR 
as a management and motivational tool. 

The current study analyzed data from 6,149 Texas schools. Depending on 
whether they did or did not own the Accelerated Reader, the campuses were 
identified either as AR or non-AR schools. Each AR school was compared with 
a cohort of non-AR schools determined by the Texas Education Agency to be 
demographically similar to that particular AR school. 

Data on academic achievement came from the Texas Assessment of Academic 
Skills (TAAS), an annual statewide testing program. We compared the TAAS 
pass rates of each AR school to the median pass rate of its non-AR cohort. The 
fundamental finding of this study is that the pass rates of AR campuses tended, 
with considerable statistical significance, to be above the median pass rates of 
their comparison cohorts. 

The higher pass rates were not limited to reading scores. Analysis showed 
statistically significant differences for reading comprehension, writing, math, 
science, and social sciences; all results favored the AR schools over the non-AR 
schools. This implies that improved reading skills developed using the 
Accelerated Reader led directly to improved general academic achievement. 

In addition to higher academic achievement, AR schools were likely to have 
higher attendance rates than their non-AR socioeconomic peers. Significantly, 
the probability of outstripping their peers rose in conjunction with the number 
of years since they had purchased the software. 
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Finally, while AR schools at every socioeconomic level outperformed their non- 
AR cohorts, die most significant results were for schools ranked low on the 
socioeconomic ladder — which is to say, AR was most successful where success 
counts the most. 

Thus, this study indicates that the Accelerated Reader software has a 
measurable positive impact on school performance. 



Previous Studies on Importance of Reading Practice 

Two Institute studies, published in 1992 and 1993, together with a third 
published in 1996, documented that in-school reading practice is a curricular 
necessity if all students are to develop into competent readers. 1 The 1996 study, 
tided Patterns of Reading Practice , was by far the largest ever to address the 
subject, with data on 659,214 K-12 students. Its results are compelling evidence 
that schools should devote far more class time — as much as an hour per day, 
from the first day of kindergarten to the last day of high school — to reading 
practice. (By contrast, students currendy spend only seven minutes per day in 
reading practice. High school students average only two to four minutes per day, 
about the same amount of time as kindergartners.) One finding was that higher 
levels of reading practice correlated to higher achievement not only in reading 
but in math as well, suggesting that strong reading skills are a foundation for all 
other academic achievement. 



A Brief Introduction to the Accelerated Reader 

The Accelerated Reader is a system of computerized testing and record- 
keeping. Its goal is to increase literature-based reading practice. This goal 
dovetails with the findings of Patterns of Reading Practice, that the more students 
read, the better they perform on academic achievement tests in both reading 
and mathematics. 

The program includes a list of more than 12,000 books, each of which has a 
reading level calculated according to a widely-used readability index. 2 Each 
book is also assigned a maximum “AR Point Value,” derived from its length and 
reading level according to the following formula: 

at)t\ • / 1 r\ d j ' t a (Words In Book) 

AR Roints = (10 + Readme Level) x 

V 6 1 100,000 



The procedure for the student is: 
1 . Select a book from a list. 
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2. Read the book. 

3. Go to the computer and take a multiple-choice, objective test about the 
book. 

The test takes the place of book reports and other evaluation tools, assuring the 
teacher that the student has read the book. The program awards the student AR 
points equal to the percentage test score times the book’s AR Point Value, 
provided that the student scores at least 60 percent on the test. The combination 
of test scores and points earned gives teachers a quick, efficient way to track 
student achievement, enabling timely, effective intervention when indicated. AR 
provides 21 different reports for teachers, students, and parents. 



Previous Studies on the Accelerated Reader 

As indicated above, there is abundant anecdotal evidence that use of the 
Accelerated Reader gets students to read more books, and that scores on 
standardized tests rise accordingly. There have been four small-scale 
independent studies that lend support to the anecdotes. 3 Two of these reports 
include control groups: Peak & Dewalt (93), and Vollands, Topping & Evans 
(96). These two controlled studies are summarized below: 



Peak and Dewalt 

In February 1993, Professor Mark W. Dewalt and Janie Peak published the 
results of a five-year longitudinal study conducted at two schools in North 
Carolina: Cherryville Junior-Senior High School in Cherryville, and Grier 
Junior High School in Gastonia. Dewalt and Peak chose the two schools because 
they were similar in student populations, minority populations, and 
socioeconomic levels. They tracked a group of 25 students from each school 
over a five-year period, using mean CAT reading scores from third-, sixth-, and 
eighth-grade cumulative records. The mean score for the Grier students was 
736 when they were tested in third grade. Their score rose to 767 in sixth grade, 
and then to 775 in eighth. Meanwhile, the Cherryville students started at 726, 
then rose to 780, and finished at 797. 

The Cherryville students, who used the Accelerated Reader for all five years of 
the study, improved their mean score by 18 points per year over the first three 
years, and 8.5 points per year over the second two years. The Grier students, 
who did not use the Accelerated Reader, saw mean improvements of only 10.3 
and 4.0 per year over each of the test periods. 

These numbers strongly suggest that use of AR helps children become better 
readers. Nonetheless, there may be factors that exaggerate the effect. For 
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example, Cherryville is a somewhat smaller school serving a more rural 
population than Grier: We do not know how this may have affected the results. 
Similarly, Cherryville may have teachers who are more dedicated to the 
literature-based reading approach that AR is intended to facilitate. 



Vollands. Topping and Evans 

The most recent study of which we are aware was performed under the direction 
of Professor Keith Topping of the Centre for Paired Learning, at Dundee 
University in Scotland. This two-part study contrasted the Accelerated Reader 
first with a standard school curriculum, and also with a labor-intensive 
alternative teaching method. The study measured student reading scores and 
attitudes toward reading, before and after several weeks in each of the above- 
mentioned programs. 

The two parts of the study were conducted in different schools using different 
students both for the AR group and the control group. Part A included 27 AR 
students and 12 control students, while Part B compared 24 students in the AR 
class with 26 students in the alternative class. 

Both parts of the study suffered problems with the implementation of AR. 
Teacher training and book availability were limited by a lack of resources. Early 
in the program, students tended to read too many books too quickly, using up the 
available supply without deriving the maximum benefit. Had the teachers been 
fully trained, one expects they would have intervened to correct this. 4 Moreover, 
this may be a general concern — that it takes a certain amount of time for 
students to become familiar with the system, and to understand how to work 
within it. This is an effect that must be planned and accounted for if one is to 
perform a successful study. 

Despite difficulties with implementing the Accelerated Reader in a short time, 
the Dundee researchers observed that use of AR gave significantly better results 
than the standard method, and was at least as effective as the more costly 
alternative method. 



The Data 

This study is based on data collected by the Texas Education Agency (TEA) in 
the spring of 1995. We chose Texas for several reasons: 

• Texas is a large, diverse state with demographics representative of the 
entire U.S. 

• Of states having comprehensive testing, Texas is one of the few of which 
we know that makes data available electronically. 
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• Texas provides a socioeconomic ranking for its schools, enabling us to 
establish control groups. 

• Texas has a high percentage of schools owning the Accelerated Reader 
(more than 40 percent). 



Demographically, Texas is reasonably analogous to the nation as a whole. It has 
a slighdy more urban population (about 80 percent, versus 75 percent for the 
nation), 5 and a slightly more racially diverse population (about 25 percent non- 
Caucasian, vs. 20 percent for the nation). 6 Since these percentages are both 
increasing nationally, however, Texas probably represents national 
demographics a few years into the future. Texas also has a higher percentage 
of persons living below the poverty line (18 percent vs. 13 percent), but as we 
shall see, our study accounts for the socioeconomic status of students on a 
school-by-school basis. 

In keeping with a legislative mandate, TEA annually conducts a comprehensive 
criterion-referenced test, the Texas Assessment of Academic Skills (TAAS). 
The tests are given in the 3rd through 8th grades, and in the 10th. All Texas 
schoolchildren in these grades take the test; some elements are administered to 
every grade, and others only at certain grades. One of the agency’s goals for the 
test is that it “include only those items judged to be free of gender, ethnic and/ 
or cultural bias, and deemed acceptable by the educator review committees.” 7 
More particularly, the test is intended to reliably assess both basic skills and 
higher-order thi nkin g skills. Toward that end, TEA has implemented an 
exhaustive procedure of quality control, based both on statistical methods and 
oversight by interested parties. The tests are normed using cumulative data from 
the 1990 administration of the test, involving millions of students. 8 

Texas makes individual schools’ pass rates for the TAAS available 
electronically, facilitating large-scale statistical analysis. Thus, other 
researchers would be able to replicate this study. 9 Texas also has developed a 
scheme for placing schools on a socioeconomic continuum, in which a campus’ 
rating derives from a weighted average as follows: 10 



Weight 

(35 percent) 
(3 5 percent) 
(12 percent) 
(12 percent) 
( 6 percent) 



Factor 

Percent Economically Disadvantaged 
Percent Minority 

Percent Limited English Proficiency 

PercentMobility 

District Wealth 



(100 percent) 



Socioeconomic rating 
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Using this scale, researchers can compare any school to a control group made 
up of schools with similar ratings. TEA defines a school’s peer group to be the 
100 schools nearest it on the scale. 

For our purposes, we needed to compare each of Texas’ 2,511 AR schools (those 
which own the Accelerated Reader) with a control group of non-AR schools 
(those which do not). By cross-referencing Advantage Learning Systems’ 
database of AR owners with the TEA database, we were able to identify 80 to 
100 peers for each AR school. 11 We then created a control peer group for each 
AR school by selecting only its non-AR peers. The resulting control groups 
generally contained 40-60 schools. 

We also used data collected by Quality Education Data, Inc., which sells data 
primarily for marketing purposes. By linking QED’s database with the TEA and 
ALS data, we were able to compare AR’s effectiveness in different categories of 
schools, such as those with many computers versus those with relatively few, and 
to search for potential biases in our sample. 



The Study 

The study started with the hypothesis that the Accelerated Reader has no effect 
on TAAS scores or attendance in Texas schools (the so-called “null 
hypothesis”). From this perspective, one assumes any observed correlation 
between AR ownership and scores or attendance is likely to have occurred by 
chance. If the null hypothesis fails to hold, we may then ask whether the effect 
is explained by some other characteristic of the data. We identified six 
particularly informative analyses: 

1 . Comparison of pass rates at AR schools with median pass rates of their non- 
AR peers 

2. As above, but broken out by years of AR ownership 

3. As above, but broken out by metropolitan status 

4. As above, but broken out by Orshansky socioeconomic status 

5. As above, but broken out by ratio of microcomputers to students 

6. Comparison of attendance rates at AR schools with median attendance 
rates of their non-AR peers 

1. From the available TEA data, we chose 30 test-score categories for analysis. 
The categories include data from grades 3 through 8, as well as grade 10. The 
test for each category was the same: Count the number of AR schools whose pass 
rate in that category was above the median pass rate for that particular school’s 
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peer cohort, as described earlier. If AR has no effect on pass rates, we would 
expect that about half of all AR schools would score above their cohort’s median, 
and about half would score below. We can evaluate discrepancies from this 
expectation by calculating a Z-value according to this formula: 12 

2 (n Above Median - 0.5) - 0.5 (N - n At Median) 

0.5 V(N^ n At Median) 

The higher the derived Z-value, the less likely it is that observed discrepancies 
are the result of chance; i.e., the more likely it is that the observations are due to 
real effects of owning the Accelerated Reader. 

The categories included reading, math, and overall pass rates for each of the 
grades tested. Other categories included pass rates for the writing test 
administered in fourth, eighth and tenth grades, and the social studies and 
science tests administered in eighth grade. The final categories were cumulative 
across all grades for reading, math, and overall pass rates, and across grades 4, 
8, and 10 for the writing test pass rates. 

2. We performed a similar analysis to that in part 1, except that we divided the 
AR schools into four classes: 

A. Owned AR for less than 1 year; 

B . Owned AR for 1 - 2 years; 

C. Owned AR for 2 - 3 years; 

D. Owned AR for 3 or more years. 

We then calculated the number of AR schools in each of these classes that had 
performed above the median of their cohort in overall pass rates, in all grades, 
and calculated Z-values as described above. We also performed chi-square tests 
across the categories to evaluate the significance of any differences observed. 

3. Exactly as in part 2, except that AR schools were divided according to their 
metropolitan status (urban, suburban, rural), identified via the QED database. 

4. Exactly as in part 2, except that AR schools were divided into three groups 
according to their Orshansky socioeconomic status (SES), as provided by 
QED. 13 Schools with Orshansky ratings less than 36 were grouped as high SES; 
those rated 36 to 51 were considered as medium SES; and those rated above 51 
were labeled low SES. 

5. Exactly as in part 2, except that AR schools were divided into three groups 
according to their ratio of students to microcomputers (high: 1-9 students per 
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computer; medium: 10-19 students per computer; low: 20 or more students per 
computer). Again, these data were obtained from QED. 

6. Our interest in attendance rates was motivated in part by informal reports 
from teachers and school administrators, observing that use of AR seemed to 
improve attendance. Since TEA provides suitable data, we included this in our 
study. Among those schools for which attendance data were available, we 
performed a similar analysis to those above, counting the number of AR schools 
whose attendance was above the median for their socioeconomic cohort, and 
then calculating a Z to measure the statistical significance of any deviation from 
the expectation that about half of AR schools would be above the median and 
about half would be below it. 

Results 

The results of this study lead us to reject the null hypothesis; that is, AR schools 
show a statistically significant tendency to perform better on the TAAS than do 
their non-AR peers. Moreover, our analysis fails to identify any biases in the data 
that would account for this observed tendency. 

In almost every category, Z scores showed a statistically significant number of 
AR schools having higher pass rates than their peers. Generally, 52 percent to 
58 percent of AR schools in a given category scored above their cohort’s 
median. 14 In most categories, Z scores indicate a statistical significance at a level 
of 1 percent or better; i.e., there is a less than 1 percent chance that an effect of 
this magnitude would arise simply by chance. Notable exceptions were scores 
for sixth-graders and tenth-graders, which showed no significant effect of AR 
ownership. Possible explanations for this are given in the discussion following 
the results of all six parts. 
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Table 1 

Use of AR Improves TAAS Pass Rates in Most Cases 



Comparison of Texas AR Schools' TAAS Pass Rates With the Median of Non-AR 


Comparison 


#AR Schools 


#AR Schools 


#AR Schools 


N 


Proportion 


Z 




With Pass Rate W/ Pass Rate 


W/ Pass Rate 




Above Median 






Below Median 


At Median 


Above Median 








All tests, 3rd grade 


685 


7 


818 


1510 


0.54 


3.40 


Reading test, 3rd grade 


705 


13 


793 


1511 


0.53 


2.25 


Math test, 3rd grade 


715 


5 


791 


1511 


0.53 


1.93 


All tests, 4th grade 


689 


10 


794 


1493 


0.54 


2.70 


Reading test, 4th grade 


648 


13 


830 


1491 


0.56 


4.71 


Math test, 4th grade 


708 


4 


780 


1492 


0.52 


1.84 


Writing test, 4th grade 


699 


4 


790 


1493 


0.53 


2.33 


All tests, 5 th grade 


632 


4 


692 


1328 


0.52 


1.62 


Reading test, 5th grade 


616 


11 


701 


1328 


0.53 


2.31 


Math test, 5th grade 


621 


2 


705 


1328 


0.53 


2.28 


All tests, 6th grade 


417 


2 


419 


838 


0.50 


0.03 


Reading test, 6th grade 


416 


6 


416 


838 


0.50 


-0.03 


Math test, 6th grade 


420 


2 


416 


838 


0.50 


-0.17 


All tests, 7 th grade 


298 


4 


355 


657 


0.54 


2.19 


Reading test, 7th grade 


294 


7 


356 


657 


0.55 


2.39 


Math test, 7th grade 


289 


1 


367 


657 


0.56 


3.01 


All tests, 8th grade 


274 


2 


378 


654 


0.58 


4.03 


Reading test, 8th grade 


282 


7 


368 


657 


0.57 


3.33 


Math test, 8th grade 


280 


3 


372 


655 


0.57 


3.56 


Writing test, 8th grade 


278 


1 


378 


657 


0.58 


3.87 


Social studies, 8th grade 


288 


3 


366 


657 


0.56 


3.01 


Science, 8th grade 


304 


2 


351 


657 


0.54 


1.80 


All tests, 10th grade 


142 


2 


138 


282 


0.49 


-0.30 


Reading test, 10th grade 


146 


1 


135 


282 


0.48 


-0.72 


Math test, 10th grade 


140 


2 


139 


281 


0.50 


-0.12 


Writing test, 10th grade 


132 


1 


149 


282 


0.53 


0.95 


All tests, grades 3-8 & 10 


1136 


15 


1360 


2511 


0.54 


4.46 


Reading, grades 3-8 & 10 


1144 


17 


1350 


2511 


0.54 


4.10 


Math, grades 3-8 & 10 


1131 


15 


1365 


2511 


0.55 


4.66 


Writing, grades 4, 8 & 10 


1073 


7 


1245 


2325 


0.54 


3.55 



Z score of 1.65 is statistically significant at 5 percent level. 

Z score of 2.33 is statistically significant at 1 percent level. 

The statistics favor the AR schools in all subject areas: Reading, math, writing, social studies and science. In the case of 
seventh-graders, the effect was slightly more pronounced on math scores than on reading scores. 

The weight of the cumulative data leaves no doubt. When summing across all grades, we see gains for AR schools in 
reading and in math, with 54 to 55 percent of AR schools performing above their peers’ median. The Z scores for these 
categories (4.10 and 4.66) indicate near certainty that the effect is real. 




Table 2 

TAAS Pass Rates Improve the Longer AR at School 

Comparison of Texas AR Schools' Pass Rates With the Median of Non-AR Peers. 

By Years of Use. 



Comparison 


#AR 


#AR 


All tests, grades 3-8 & 10 


< Median 


at Median 


Owned AR < 1 year 
All tests, grades 3-8 & 10 


408 


5 


Owned AR 1-2 years 
All tests, grades 3-8 & 10 


346 


3 


Owned AR 2-3 years 
All tests, grades 3-8 & 10 


185 


1 


Owned AR 3 or more years 


197 


6 


Totals: 


1136 


15 



#AR 

> Median 


N 


Proportion 
Above Median 


Z 


395 


808 


0.49 


-0.49 


420 


769 


0.55 


2.64 


263 


449 


0.59 


3.64 


282 


485 


0.59 


3.84 


1360 


2511 


0.54 


4.46 



Analysis of all scores over all grades shows that AR has no significant effect on test results during its firstyear of ownership. (AR 
schools actually performed somewhat below the median, but with no statistical significance.) There is a substantial gain during 
the nextyear, and once more the following year. At this point the effect appears to stabilize. Schools that have owned AR for 
2 or more years are 59 percent likely to have pass rates above the median for their control group. We calculated a Chi-square 
value of 16.05, which indicates a p-value <0.005. 



Table 3 

AR has greatest impact on urban schools 

Impact of AR on TAAS Pass Rates by Metro Status 



All tests, grades 3-8 & 10 


#AR 

< Median 


#AR 

at Median 


#AR 

> Median 


N 


Proportion 
> Median 


Z 


Metro Status = 1 (Urban) 


379 


2 


540 


921 


0.58 


5.2779 


Metro Status = 2 (Suburban) 


468 


12 


524 


1004 


0.52 


1.7463 


Metro Status = 3 (Rural) 


289 


1 


296 


586 


0.50 


0.2481 


Totals: 


1136 


15 


1360 


2511 


0.54 


4.4636 



Analysis of AR’s relative effectiveness in different metropolitan settings indicates that AR is by far most influential in urban, 
rather than rural or suburban schools. Fifty-eight percent of urban AR schools had pass rates above their cohort’s median, with 
a Z value of 5.28. Fifty-two percent of suburban AR schools were above average, while rural schools showed no statistically 
significant difference in pass rates. Chi-square analysis of these differing effects gave a result of 1 1.6 (p < .005). 



Table4 

AR Has Greatest Impact on Low SES Schools 

Impact of AR on TAAS Pass Rates by SES 



All tests, grades 3-8 & 10 


#AR 


#AR 


#AR 


N 


Proportion 


Z 


< Median 


at Median 


> Median 




Above Median 




Orshansky = 0-35 (upper) 


355 


4 


415 


774 


0.53 


2.1262 


Orshansky = 36-51 (middle) 


467 


7 


484 


958 


0.50 


0.5188 


Orshansky = 52-99 (lower) 


314 


4 


461 


779 


0.59 


5.2445 


Totals: 


1136 


15 


1360 


2511 


0.54 


4.4636 



Analysis of AR’s relative effectiveness in different socioeconomic settings indicates that AR is most useful in low SES 
environments. Fifty-nine percent of low SES schools (52 <= Orshansky <= 99) had pass rates above their cohort’s median, 
with a Z value of 5.24. Chi-square: 12.9 (p < .005). 



Tables 

AR Is Effective Regardless of Computer Availability 

Impact of Computer Microdensity on TAAS Pass Rates 



All tests, grades 3-8 & 10 


#AR 

< Median 


#AR 

at Median 


#AR 

> Median 


N 


Proportion 
Above Median 


z 


Microdensity = low 
20+ students per computer 


222 


3 


267 


492 


0.54 


1.9897 


Microdensity = medium 
10-19 students per computer 


437 


8 


518 


963 


0.54 


2.5887 


Microdensity = high 
1-9 students per computer 


477 


4 


575 


1056 


0.54 


2.9906 


Totals: 


1136 


15 


1360 


2511 


0.54 


4.4636 



Analysis indicates that AR is effective regardless of the relative availability of microcomputers within the school. Having 
categorized schools as having 20 or more students per computer, 10 to 19 students per computer, or 1 to 9 students per 
computer, we found that in each group, 54 percent of AR schools placed above the median TAAS pass rate of their TEApeers. 
Chi-square: 0.14 (not significant). 



Table 6 

AR Improves Attendance 

Comparison of Texas AR Schools' Attendance Percent With the Median of Non-AR Peers 

Comparison #AR #AR #AR N Proportion Z 

< Median at Median > Median Above Median 

Campus attendance rate 343 47 540 930 0.61 6.60 

The largest effect of any discovered in this study was that which AR appears to have on attendance. Sixty-one percent of 
AR schools showed higher attendance rates than the median of their peer controls. Although there was a smaller sample 
than in other parts of the study, because of limited data availability, the Z value for this result was nonetheless 6.60. 
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Discussion 

Significance of results 

Despite the significance indicated by the high Z values, one might be tempted to 
dismiss these results because of the relatively low magnitudes of the effects 
observed. It is important to understand that we are wielding a rather blunt statistical 
instrument. In a sense, we have set up a detector at the passing score of each test, 
and we detect only students who cross that line due (we hypothesize) to their 
schools’ usage of AR. 15 We know nothing else about their scores, nor those of 
their classmates. Given this constraint, our observation of statistically 
significant results suggests that there is an iceberg of other effects that we 
cannot measure. 

In particular, Paul (92) observed that the greatest effect of additional reading 
practice is on those students with the lowest reading ability. Such students may 
be earning much higher scores at AR schools than at non-AR schools, but we will 
not observe them unless their improvement pushes them across the pass 
threshold — a comparatively unlikely event given their starting point. Thus, for 
every raised score we detect, there should be many, many more that we can- 
not observe. 

Similarly, we observed no significant effect for AR in rural schools. However, 
the median overall pass rate for third through fifth grades was 68.2 for rural AR 
schools versus 63.5 for urban AR campuses. One implication is that the urban 
schools had more room for measurable improvement, because they had a larger 
percentage of students who could conceivably “pass through” our detector. 

Eliminating bias concerns 

The results of Part 1 clearly demonstrate a correlation in the data between AR 
ownership and higher test scores; thus, we reject the null hypothesis. However, 
a critic may suggest alternatives to the conclusion that AR causes these higher test 
scores. In particular, we must look for potential sources of bias in the data. 

Self-selection bias 

One might argue that because AR schools have a presumed bias towards 
emphasizing reading skills — otherwise, why would they buy the software? — 
these schools would do better whether or not they owned the software. The 
results of Part 2 tend to refute this. The relationship between length of 
ownership and observed effect is important because it tells us that this self- 
selection bias does not compromise our other results. AR schools do no better 
than their peers until they have had time to implement the software into 
their curriculum. 

Indeed, although we present further analyses that eliminate other bias concerns, 
this one result is the most telling. If there were any particular characteristic that 
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made a school more likely to own AR, and that similarly made that school more 
likely to perform well on the TAAS, we would expect first-year AR schools to 
show above-average results. They do not. 

Rural/urban bias 

One might argue that AR schools are likely to belong to a metropolitan category 
that performs better than its socioeconomic peers. However, we found that 38 
percent of all Texas schools fall into our urban classification, while 37 percent 
of Texas’ AR schools do. 16 Our AR sample is slightly biased towards suburban 
schools (40 percent, versus 38 percent for all of Texas), which should not cause 
significant variations in our results. 

Technology bias 

One might also argue that since AR schools are more likely to own computers, 
they are also more likely to be taking advantage of other software that 
contributed to the apparent effect of AR. Our data do not support this claim. As 
shown in Part 5, we observed no relationship between the ratio of computers to 
students and the apparent effectiveness of AR. 

Socioeconomic bias 

This potential source of bias is primarily addressed by the use of the TEA control 
groups. As observed above, we ran hypothesis tests to assure that the TEA 
groups are indeed statistically unbiased with respect to pass rates. 

One other source of bias 

The major bias in this study should attenuate, rather than exaggerate, the 
observed effects of AR. This bias occurs because we classify any school that 
owns AR as being an AR school, although we do not know to what extent they 
are using the software; indeed, we do not know whether they are using it at all. 
In our analyses, a school in which one of ten English teachers uses AR, perhaps 
with selected classes or populations, has equal weight with an AR school in 
which AR is used in all classes by all students in all grades. The result of Part 2 
is strong evidence that this will bias the study towards the null hypothesis, since 
schools that own the program but haven’t yet had time to use it show no 
significant improvement over the performance of their control groups. From 
this perspective, the recent success of AR in Texas schools works against it: 
almost one third of the AR schools bought the software during the year the data 
were collected. The simple expedient of removing these schools from the sample 
altogether would have pushed the results much further in favor of AR’s 
usefulness. Including them as non-AR schools would have been even more 
dramatic in its effect on the numbers. To claim that there is no causal 
relationship between AR ownership and higher pass rates is to assert that 
somewhere in the data lurk unknown biases that are powerful enough to 
overcome this huge negative bias. 
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The Biggest Benefit 

The results of Part 4 indicate that AR has its strongest effects in schools at the 
low end of the socioeconomic scale. More precisely, we observe that AR schools 
with low socioeconomic ratings are more likely than any others to outperform 
their non-AR cohorts. 17 We may infer from this the group of students most likely 
to benefit from AR: Socioeconomically disadvantaged children. This 

corresponds to our general understanding. We know 18 that children in these 
schools are least likely to come from homes where reading is a valued activity, 
and that such children are similarly least likely to become readers themselves. 
They thus have the most to gain from an instructional tool designed to moti- 
vate reading. 

Anomalous Results 

As noted above, AR schools showed no statistical difference from their non-AR 
cohorts for tests in sixth and tenth grades. We can offer some explanations for 
these observations. 

Since many students switch from one elementary school to a different middle 
school at sixth grade, we expect that a shuffling will happen in which some 
students move from AR campuses to non-AR campuses, and vice versa. Because 
we are dealing with snapshot data, there is no way for us to track these students. 
This blurring of student groups is itself enough to explain the anomalous result; 
however, we also know (as demonstrated in Part 4) that the longer a school owns 
AR, the better that school’s scores are likely to be. It may be that part of the 
advantage lies with the students’ familiarity with the AR system. As noted earlier, 
Vollands et al (96) observed that when first introduced to AR, students tend to 
read too many books too quickly. 

The anomaly in 10th grade may simply be that although these schools own AR, 
they do not use it to any great extent. Our observation is that high schools 
typically use AR in a limited way, often for remedial purposes or as an optional 
supplement to the regular English curriculum. As noted above, lack of usage 
data inherently biases this study towards the null hypothesis. 

Attendance 

Overall, the data leave little doubt about the influence AR can have in schools. 
Of all the results provided, none is so telling as that concerning attendance. To 
the extent that students who are successful in school are more likely to go there, 
if AR did nothing but give children one subject in which they considered 
themselves competent, we believe it would make them more likely to attend. We 
can identify several direct ways in which AR motivates students to attend school: 

• Must attend to take the tests. 
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• Facilitates social benefit of reading; i.e ., discussions with peers about books 
one has read. 

• School library has the books. 

These first-order effects will inevitably influence a host of higher-order 

patterns. Here is a list of some possible causal relationships: 

• Success in reading motivates students to attend; increased attendance 
brings success in other subjects. 

• Success in reading raises self-esteem; increased self-esteem motivates 
students to try harder in other subjects; trying harder brings success in 
other subjects. 

• Success makes students better readers; being better readers lets them 
understand texts in other subjects. 

• Success in reading improves students’ cognitive skills; improved cognitive 
skills help students learn other subjects; etc. 



Conclusions and Policy Implications 

The data show that schools with AR outperform their peers on standardized 
tests in all subject areas. They do not measure the effect size, largely because of 
the inability to gauge AR usage, but the implication is that AR is more influential 
— possibly much more so — than these analyses can demonstrate. One might 
note the small magnitude of the effect (54 percent above the median, versus 50 
percent expected) and dismiss the results as too small to be important. This 
would be an inappropriate interpretation. As the discussion above makes clear, 
the study is strongly biased against showing a positive result for AR. For schools 
owning AR more than two years, the figure is 59 percent above the median; and 
this still does not take into account the degree to which AR has been 
implemented. 

It is beyond the scope of this study and of the currently available data to discover 
more precise and useful relationships, such as the likelihood that a student who 
would otherwise fail a particular TAAS test would instead pass if the student’s 
school included AR as part of a fully implemented curriculum of literature-based 
reading practice. We hope that in the future, automated data-collection 
techniques embedded within products such as the Accelerated Reader and 
S.T.A.R.™ (a computerized reading assessment system released by ALS in fall 
of 1996) will enable more sophisticated and precise measurements of the 
effectiveness of instructional tools and techniques. 
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Nonetheless, together with the results of Patterns of Reading Practice (as well as 
other previous Institute studies) showing the value of reading practice in 
improving overall academic achievement, it is reasonable to assert that AR is an 
effective tool in stimulating increased reading, and that increased reading will 
lead to higher attendance rates and greater academic success. 

Not incidentally, these benefits will accrue primarily to those who need them 
most: socioeconomically disadvantaged children living in urbanized areas. This 
fits well with the concept of value-added education: Achieving results where 
they make the greatest difference. 

We can project many benefits beyond the obvious economic opportunities that 
come with academic success. Children who read literature understand that there 
is a world beyond the immediate horizon (or concrete wall); that their own 
circumstances do not necessarily represent normalcy; that there are values and 
possibilities more uplifting and universal than those found on the street; and that 
there is something called morality which we must all address in our choices of 
how to live. 
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Notes 

1. Paul, T. (1992). 1992 National reading study and theory of reading practice. 
Madison, WI: The Institute for Academic Excellence. Paul, T. (1993). National 
study of literature-based reading: How literature-based reading improves both reading 
and math. Madison, WI: The Institute for Academic Excellence. Paul, T. 
(1996). Patterns of reading practice. Madison, WI: The Institute for Academic 
Excellence. 

2. Since January 1994, Advantage Learning Systems, the publisher of the 
Accelerated Reader, has used an automated program which utilizes the Flesch- 
Kincaid reading index to determine readability. See Flesch, R. (1974), The art 
of readable writing with the Flesch readability formula. New York: Harper & Row; 
and Kincaid, J.P., et al. (1981, March). (Computer reading ability editing 
system, TEEE Transactions on professional communications .) Prior to 1994, 
Advantage Learning Systems used the Fry Readability Index. See Fry, E. (1968). 
“A readability formula that saves time.” Journal of Reading, 11, 513-516. 

3. Peak, J. & Dewalt, M. (1994). “Reading achievement: Effects of 
computerized reading management and enrichment.” ERS Spectrum , 12:31-34. 

Turner, T. (1993). Improving reading comprehension achievement of sixth, seventh , 
and eighth grade underachievers. Nova University. 

McKnight, D. (1992). Using the Accelerated Reader and other strategies and varied 
techniques to improve the reading attitudes of fifth grade students. Nova University. 

Vollands, S., Topping, K., & Evans, H. (1996). Experimental evaluation of 
computer assisted self-assessment of reading comprehension: effects on reading 
achievement. Presented at the National Reading Research Center Conference 
on Literacy and Technology, October 4, 1996, Atlanta, Georgia. 

4. Recall that students can only get points for books for which AR includes a test. 
Moreover, students may only test once for a given book. If students read too 
quickly, they score poorly because they aren’t reading with comprehension. 
When implemented according to design, teachers oversee students’ reading 
patterns, and if their test scores are too low, intervene with advice on reading 
level and rates. 

5. The sourcebook of ZIP code demographics , Vol. 1. (309-A) (1992). Arlington, VA: 
CACI Marketing Systems. 

6. The sourcebook , (307-B). 

7. Texas student assessment program technical digest for the academic 1994-95 
(1995). Austin: Texas Education Agency. 
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8. Texas student assessment program Chapter 8. 

9. Advantage Learning Systems will make available its Texas customer list on 
a confidential basis to qualified researchers who may wish to validate the study 
results. 

10. The unidimensional ranking for a school is a group index calculated in three 
steps: First, a standard Z score is computed for each of the five demographic 
variables, reflecting the significance of that variable for the particular school; 
second, these standard scores are multiplied by their weights as shown in the 
table in the text; finally, these values are summed to provide the school’s group 
index. 

• Economically Disadvantaged students are those eligible for free/reduced- 
price lunch or other public assistance. 

• Minority students are non-white (African American, Hispanic, Asian, 
Pacific Islander, Native American). 

• Limited English Proficient are those identified by the Language 
Proficiency Assessment Committee or designated professional. 

• Mobile students are those who have been in membership at the school for 
less than 83 percent of the school year. 

• District Wealth is a ranking from 1-10 based on the ratio of total taxable 
property value to the number of students in the district. 

11. To demonstrate that the TEA defined control groups are unbiased with 
respect to TAAS performance, hypothesis tests were run for each of the 30 test 
and test-composite pass rate categories for which data are provided on the TEA 
WWW server. None of these comparisons were statistically significant in their 
deviation from the expectation that 50 percent of the schools would have pass 
rates higher than the median of their control group, and 50 percent would have 
pass rates that were lower. 

12. Daniel, W. (1990). Applied nonparametric statistics , second edition . Boston: 
PWS-KENT Publishing Company. Chapter two discusses the “One-sample 
sign test” technique, including the use of the normal approximation to the 
binomial to generate p-values. 

13. Orshansky Percent is the number of students below the federal poverty line, 
given as a percentage of all children within the district’s boundaries. 

14. Note that calculation of the proportion above/below the median does not 
incorporate campuses at the median. This is consistent with the calculation of 
the Z-statistic, which specifically measures the likelihood of a given distribution 
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above and below the median occurring due to chance, with events at the median 
being ignored. See note 12 for references on the Z-statistic. 

15. We must emphasize that this is a metaphor. We cannot know whether any 
individual student would or would not have passed any particular exam, 
regardless of AR ownership at school. 

16. We found that rural AR schools had higher mean pass rates than urban AR 
schools, yet in many categories they were significantly likely to have pass rates 
below the median for their non-AR peers. One explanation is that with 
socioeconomic factors held equal, urban schools in general may outperform 
rural schools. 

1 7. Some of this effect is due to our technique. Low-SES schools generally have 
lower pass rates (e.g., for AR schools, mean pass rates on third-, fourth- & fifth- 
grade reading tests were 73 percent for low-SES versus 86 percent for high-SES 
campuses). Thus, low-SES schools have more students who can potentially 
improve and cross the pass line. 

18. Watkins, M. & Edwards, V. (1992). “Extracurricular reading and reading 
achievement: the rich stay rich and the poor don’t read.” Reading Improvement , 
29, 236-242. Goodlad, J. (1984). A place called school. New York: McGraw-Hill. 
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